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ABSTRACT 

In an experiment conducted under classroom conditions 
with 8th grade students, the simulation game "Democracy" was shown to 
be effective at teaching that "log-rolling" is an acceptable part of 
the legislative process. Two aspects of the game — role identification 
and gcune structure — were shown to contribute independently to its 
effectiveness. However, the game proved ineffective in changing 
political attitudes not directly related to log-rollipg. (Author) 
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INTRODUCTORY STATEMENT 



The Center for Social Organization of Schools has two primary 
objectives: to develop a scientific knowledge of how schools affect 

their students, and to use this knowledge to develop better school 
practices and organization. 

The Center works through five programs to achieve its objec- 
tives. The Academic Games program has developed simulation games 
for use in the classroom. It is evaluating the effects of games 
on student learning and studying how games can improve interpersonal 
relations in the schools. The Social Accounts program is examining 
how a student's education affects his actual occupational attainment, 
and how education results in different vocational outcomes for blacks 
and whites. The Talents and Competencies program is studying the 
effects of educational experience on a wide range of human talents, 
competencies, and personal dispositions in order to formulate -- 
and research — important educational goals other than traditional 
academic achievement. The School Organization program is currently 
concerned with authority-control structures, task structures, 
reward systems, and peer group processes in schools. The Careers 
and Curricula program bases its work upon a theory of career devel- 
opment. It has developed a self-administered vocational guidance 
device to promote vocational development and to foster satisfying 
curricular decisions for high school, college, and adult populations. 

This report, prepared by the Academic Games program, examines 
two aspects of a simulation game to determine their independent 
contributions to the game's effectiveness. 
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INTRODUCTION 



For several years, a small group of educational researchers have been 
investigating the educational effectiveness of simulation games. The findings 
of this research have been mixed, although simulation games generally seem to 
be more successful at changing students' af^titudes than at teaching knowledge 
or skills.^ 

A recent study by Livingston and Kidder (1972), using the Democracy 
game (Coleman, 1969), represents an attempt to progress beyond the question, 

"What objectives does the game accomplish?" to the question "What 
characteristics of the game are responsible for its effectiveness?" Previous 
research (Livingston, 1972) had Indicated that the Democracy game is particularly 
effective at teaching students to accept "log rolling" (i.e. exchange of support 
by legislators) as part of the legislative process in a democracy. 

Livingston and Kidder subdivided this game into two major components, which 
they called "game structure" and "role identification." Game structure 
included such things as the scoring system and the order of play, which 
represent the game designer's attempt to reproduce what he considers to be the 
main incentives and constraints that guide the behavior of real congressmen. 

"Role identification" included all those features of the game that inform the 
player that his role is that of a congressman. The results showed tl at both 
game structure and role identification contributed to the effectiveness of the 
game . 



These findings are briefly summarized in a forthcoming book by 
Livingston and Stoll (1972, Ch. 5). Some of the earlier studies are reported 
in detail by Boocock and Schild (1968) . 
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The Demoerr ~y game materials actually contain rules and equipment for 
eight games, all of which are really variations of a single basic game. This 
basic game is identified in the kit as "Game 1: Legislative Session." In 
this paper the phrase " Democracy game" will be used to refer to this basic game. 



The experiment by Livingston and Kidder (1972) appears to be the first in 
which role identification and game structure were investigated as components 
of a simulation game known to be effective at achieving a specific educational 
objective. However, there has been at least one study (Fennessey, et al., 
1972) in which the investigators used an existing role-play exercise that had 
not been experimentally tested for effectiveness and added to it the structure 
of an appropriate simulation game. Classes who played the game were compared 
with classes who played the original role-play exercise and with classes who 
received instruction in the same subject by other, more conventional methods. 
The results showed no significant differences between treatment groups, 
despite the large size of the sample (60 classes; 1, 874 students). 

The experiment reported in this paper was basically a replication^ of the 
experiment by Livingston and Kidder (1972) , although it differed from the 
original experiment in two important ways. First, the subjects were 8th 
graders, rather than 10th and 11th graders. Second, this experiment was ad- 
ministered under typical classroom conditions; the earlier experiment was 
not. On the basis of the earlier experiment, both game structure and role 
identification were expected to make a significant positive contribution to 
the effectiveness of the Democracy game. 



The replication was of the type that Lykken (1968) calls "constructive 
replication," rather than "literal replication" or "operational replication." 
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METHOD 



The experimental treatments 

The experiment employed four treatments. One of the experimental 
treatments ("game plus role") was the Democracy game itself. In the 
game, each player takes the role of a congressman. He receives a set of • 
cards that indicate the number of votes toward his re-election that he will 
gain or lose according to Congress* action on each issue. The rules in- 
clude a specified sequence of events, with provision for speeches, 
"bargaining" (i.e. log-rolling), and roll-call votes on the issues. 

Two other experimental treatments were incomplete versions of the 
Democracy game. One of these ("game only") was an abstract simulation 
game in which all references to politics and legislation were removed, 
while the game structure was left intact. The rules were adapted, 
phrase by phrase, from those of the Democracy game by removing all political- 
terms. The issues were replaced by "group choices," identified only by 
letters of the alphabet; a roll call vote became a "group decision," and so 
on. The other incomplete version ("role only") was an unstructured role- 
play exercise based on the issues in the Democracy game. Each player 
received a profile card identifying him as a congressman from a 
psi^ticular type of district and describing his constituents' interests. 

There was no step-by-step procedure. Instead, the rules specified only 
that "Issues may be voted on immediately) after debate) cr after a 
recess." 

The control treatment ("no game, no role") was a simulation game that 
was assumed to be irrelevant to tb i irtudents* political attitudes, since 
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it had nothing to do with politics or group decisions.^ 



These four treatments form a 2 x 2 factorial experiment, in which the 

factors are the presence or absence of the game structure of the Democracy 

game and the presence or absence of the identification of the player's role 

2 

as that of a congressman. 



This pme was Tra_de and Develop (Livingston, 1969), which simulated economic 
growth in an international economy. 
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The experiment by Livingston and Kidder (1972) also included a fifth treat 
ment — a structured role-play exercise that included the step-by-step 
procedure of the Democracy game but not the scoring system. This version 
used the same profile cards as the unstructured role-play. 



The dependent variables 




The main dependent variable in this experiment was the students' 
acceptance of the practice of log-rolling by Congressmen, measured by the 
following three-item scale: 



Suppose two groups of Congressmen make an agreement: 
"You vote for our bill and we'll vote for your bill," 















Yes 


No 


Do 


you 


think 


this 


is 


unfair? 


( ) 


( ) 


Do 


you 


think 


this 


is 


undemocratic? 


( ) 


( ) 


Do 


you 


think 


this 


is 


dishonest? 


( ) 


( ) 



A second dependent variable was the students' belief in the prevalence 
of log-rolling in Congress. This variable was measured by a single item, 
placed directly after the three items on acceptance of log-rolling. The 
item simply asked, "How often do you think agreements like this are made 
in Congress?" The students' options were "Very often," "Fairly often," 
"Sometimes," "Occasionally," and "Almost never." 

A third dependent variable was the students' political efficacy — 
the belief that they can understand and influence the political process. 
Political efficacy was measured by a four-statement scale: 



Ordinary people can influence the Government if 
they work at it. 



Sometimes politics and government seem so 
complicated that a person like me can't really 
understand what's going on. 

The average person can't do much about politics 
and government. 



I think I understand politics fairly well. 

The response options for each statement were "I definitely agree," 
"I tend to agree," "I tend to disagree," and "I definitely disagree." 
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A fourth dependent variable was the students' intention to participate 
in the political process, measured by a four-item scale: 

Do you intend to register to vote as soon 
as you are 18? 

Would you write a letter to your Congressman 
telling him how you felt about a political issue? 

Would you work as a volunteer in an election 
campaign (handing out leaflets, and so on) for a 
candidate you favored? 

Would you contribute money to the campaign fund 
of a candidate you favored? 

The response options for these questions were "Yes, definitely," 
"Probably," "l might," "Probably not," and "Definitely not." 

A fifth dependent variable was the students' knowledge of the names 
of their representative and Senators in the United States Congress. This 
measure imficati d the extent to which the experimental treatments motivated 
the students co acquire this information during the two full days between 
their first exposure to the games and the administration of the 
questionnaire. The effect of the students' previous knowledge was controlled 
by the random assignnient of subjects to treatments. 

Subjects and procedure 

The subjects were 141 students in four eighth grade classes at a 
junior high school in a northern Chicago suburb. They were assigned 
randomly within classes to the four treatments, producing a 2 x 2 x 4 
randomize d-b lock design. The students played the three games and the 
role-play exercise in their regular social studies classes for two days, 
forty minutes each day. On the third day they answered the questionnaires. 



All the activities were administered by the regular social studies teacher. 
Thus, during each of the four classes, one teacher was supervising four 
different group activities at the same time. The experimenter was not 
present at any time during the experiment. 
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RESULTS 

Table 1 shows the results for the main dependent variable, the students' 
acceptance of log-rolling as part of the legislative process. Figure 1 shows 
the means and 957» confidence intervals for the four treatment groups, (llie 
scores on this variable and on the other three attitude variables have been 
linearly transformed so that +1,00 represents the maximum possible score, 

-1.00 represents the minimum possible score, and 0.00 represents a neutral 
position.) The differences between treatment group means are large and 
in the expected direction; the game-plus -role group is highest and the 
no-game, no-role group is lowest. Analysis of variance shows both the 
game and role factors significant beyond the .001 level, accounting for 
10 per cent and 11 per cent of the total variance, respectively. 

Table 2 presents the results for the students' belief in the 
prevalence of log-rolling in Congress. Figure 2 shows the means and 957„ 
confidence intervals for the four treatment groups. The results for this 
variable are similar to those for acceptance of log-rolling, but the effects 
are not as strong; the treatment factors account for only about 4 per cent 
and 5 per cent of the total variance. 

Table 3 shows the results for political efficacy. The differences 
between treatment groups are small and the effects do hot approach statis- 
tical significance. 

Table 4 shows the results for the students' intention to participate 
in the political process. Again, the treatment effects are not significant. 
However, there is a significant treatment x class interaction accounting for 
about 7 per cent of the total variance. 
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Table 5 shows the results for the students' knowledge of the names of 
their U.S. Senators and representative. These scores are expressed as a 
simple percentage; that is, a student who knew all three names received a 
score of 1.00, and so on. The results for this variable are somewhat 
surprising. The treatment group differences are exactly the opposite of 
those predicted, and there is a significant role x class interaction that 
accounts for 13 per cent of the total variance. Figure 3 shows this 
interaction graphically. 

Table 6 shows the intercorrelations of the five dependent variables. 
The figures above the main diagonal are the correlations computed over the 
entire sample, without respect to treatment group or class membership; 
those below the main diagonal are the withln-cell correlations, pooled over 
all 16 cells in the design. The numbers on the main diagonal are the 
internal-consistency estimates (coefficient alpha). ^ None of these 
correlations is large enough to indicate a substantial proportion of common 
variance between two variables, although the correlations are undoubtedly 
attenuated by the low internal consistency of the questionnaire scales. 
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Coefficient alpha is a version of formula KR-20 that does not require the 
items to be dichotomous. 



DISCUSSION 



The co-ordinator's manual for the Democracy game states (p,A)* 

In playing the game, the Legislator finds that if he is 
to aicceed -- that is, if he is to satisfy a majority of hi® 
constituents and thus be re-elected -- he must carry out nego- 
tiations, exchanges, and make agreements with other Legislators. 

He must be willing to make any agreement that will sacrifice 
the issues least important to his constituents if the agree- 
ment will strengthen his power over issues most important to 
IS constituents. This activity is most necessary in a legis- 
lature, but it is often viewed with suspicion by citizens. 

One of the most important things learned from the game 
is that this kind of negotiation and exchange is necessary if 
the Legislator is to do the best job for his constituents. 

The results of this experiment show clearly that the Democracy games does 

accomplish this objective and that both the role identification and game 

structure present in the game contribute substantially to its succes.s. 

The results also show the effects of the Democracy game to be quite specific. 

IJhile the game succeeded in teaching that log-rolling is an acceptable part 

of the legislative process, it did not succeed in building the students' 

political efficacy or in stimulating them to want to participate in the 

political process - not even to the extent of finding out the names of their 
own congressmen. 

As a replication, this experiment corroborates the findings of the 
earlier experiment by Livingston and Kidder (1972) . Those findings have 
now been reproduced under classroom conditions. The question that remains 
is whether they will generalize to other simulation games with other 
educational objectives. If so, they will have important implications for 
developers and users of social studies curriculum materials. They imply 
that the players' roles in a simulation game should be clearly identified 
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if the game is to attain its maximum effectiveness at teaching the players 
about the behavior of people in the real-world situation that the game 
represents. They also imply that, insofar as this bahavior is rational, 
a true simulation game is likely to produce better understanding of it 
than a role-playing exercise. This distinction is an important one, 
though it is sometimes ignored by publishers of social studies materials. 

A true simulation game contains rules and (usually) a scoring system that 
correspond to the constraints and incentives that guide behavior in the 
real-world situation. The player in the game chooses a certain behavior, 
not because he thinks it is what his real-world counterpart would do, 
but because he sees that is is advantageous for him in the game-- and 
thus also for his counterpart in real life. The reason the player's role 
should be clearly identified is to enable him to see the correspondence 
between the game and the real-world situation; between his own behavior 
and that of his real-world counterpart. 

One feature of the data in this experiment fails to support this 
interpretation -- the absence of a significant positive game x role 
interaction. If both role identification and game structure are necessary 
for the effectiveness of the game, the students in the role-only and game- 
only treatments should have scored considerably lower than they did -- 
closer to the control group than to the role-plus-game group. Neverthe- 
less, the above interpretation may still be correct, because the conditions 
of the experiment may have raised the scores of these two groups in three 
ways . 

First , experimental "contamination" could have raised the scores 
of both the role-only and game-only treatment groups. Because all four 

11 



15 



treatments were administered in the same classroom at the same time, the 
students in the role-only group may have observed and copied the log- 
rolling behavior of the game-only and the role-plus-game groups. Similarly, 
the students in the game-only group may have overheard and adopted the poli- 
tical terms used by the role-only and the role-plus-game groups. The 
students in the control group would be less likely to be affected, because 
the log-rolling behavior and the political vocabulary of the other groups 
were irrelevant to the game they were playing.^ 

Second, the role-only treatment may not have been "pure." That is, 
it may have contained one important element of the game structure of the 
Democracy game. The player's goals may have been implied by the profile 
cards, which contained such phrases as "Your constituents want ..." and 
"Your constituents are concerned about ..." These statements on the cards 
may have prompted the players to engage in some log-rolling, though 
probably much less than in the game-only and the role-plus-game groups, 
who were explicitly instructed to bargain for votes. 

Third, the questionnaire used to measure the effect of the game may 
have been a reactive measure for the game-only group. That is, the 
political context of the questions and their closeness in time to the game 
may have led some of the students in the game-only group to draw the 
analogy between their behavior in the game and the behavior of legislators 
in real life. In this way the questionnaire may have had the same effect 
as a follow-up discussion. However, the difference between the scores 

^ Some evidence for this explanation is provided by the earlier study by 
Livingston and Kidder (1972) , which was administered under conditions 
that would make this kind of "contamination" less likely to occur. 

The results of that experiment showed a small positive interaction that 
fell just short of statistical significance. It accounted for 1.7 
percent of the total variance, as compared with 5.3 and 8.4 per cent 
for the game and role effects, respectively. 
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of the game-only group and the role-plus-game groups suggests that many of 
the students in the game-only group did not draw the analogy between their 
"Group Decision" game and the United States Congress. 
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Figure 1. Acceptance of log-rolling:means and 95% confidence 
Intervals . 
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Figure 2, Belief in prevalence of log-rolling, 
and 957. confidence Intervals. 
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Figure 3. Knowledge of names of own congressmen: role x class 
interaction. 



Table 1 



Acceptance of "Log-rolling" 





Means 


(combined across classes) 




No Game 


Game 


Combined 


No Role 


-.70 


-.06 


-.36 


Role 


-.06 


+ .45 


+ .19 


Combined 


-.36 


+ .19 


+ .08 


Standard 

Internal 


deviation, within cells (pooled 
consistency (alpha) = .84 


variance) = 


.74 



Source 




Analysis of variance 






df 


SS 


MS 


F 


P 


y\ ^ 


Game 


1 


10.56 


10.56 


19.36 


.001 


.100 


Role 


1 


11.62 


11.62 


21.31 


.001 


.110 


Classes 


3 


4.13 


1.38 


2.53 


.06 


.039 


GxR 


1 


0.28 


0.28 


0.51 


N.S. 




GxC 


3 


3.64 


1.21 


2.23 


N .S. 




RxC 


3 


1.65 


0.55 


1.01 


N.S. 




vTxRxC 


;J 


5.29 


1.76 


3.23 


N.S. 




Within Cells 


125 


68.19 


0.546 




.025 


.050 


Total 


140 


105.37 
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Table 2 



Belief in Prevalence of **Log-rolling** 




Means (combined across 


classes) 


No Role 


No Game Game 

-.18 +.01 




Combined 
- .08 


Role 


+.04 +.34 




+ .19 


Combined 


-.07 +.17 




+ .06 


Standard deviation, within cells (pooled variance) = 


.30 




(No internal 
measured by 


consistency estimate is possible, since 
a single questionnaire item.) 


this 


variable was 



Source 




Analysis of variance 






d£ 


ss 


MS 


F 


P 




Game 


1 


2.01 


2.01 


5.53 


.025 


.038 


Role 


1 


2.71 


2.71 


7.45 


.01 


.051 


Classes 


3 


0.55 


0.18 


0.50 


N.S. 




GxR 


1 


0.09 


0.09 


0.24 


N.S. 




GxC 


3 


0.30 


0.10 


0.28 


N.S. 




RxC 


3 


1.20 


0.40 


1.10 


N.S. 




GxRxC 


3 


0.70 


0.23 


0.64 


N.S. 




Within cells 


125 


45.49 


0.091 








Total 


140 


53.05 
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Table 3 



Political Efficacy 







Means 


(combined 


across 


classes) 




No 


Game 


Game 


Combined 


No Role 


- 


.07 


+ • 


07 


+ .01 


Role 




.00 


• 


00 


.00 


Combined 


- 


.03 


+ . 


04 


.00 


Standard 


deviation, within cells (pooled) - .30 






Internal 


consistency (alpha) 


.35 












Analysis of variance 




Source 


df 


ss 


MS 


F 


P 


Game 


1 


0.17 


0.17 


1.92 


N.S. 


Role 


1 


0.001 


0.001 


0.01 


N.S. 


Classes 


3 


0.56 


0.19 


2.09 


N.S. 


GxR 


1 


0.16 


0.16 


1.81 


N.S. 


GxC 


3 


0.19 


0.06 


0.70 


N.S. 


RxC 


3 


0.35 


0.12 


1.32 


N.S. 


GxRxC 


3 


0.11 


0.04 


0.41 


N.S. 


Within Cells 


125 


11.11 


0.089 






Total 


140 


12.64 
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Table 4 



Intention to Participate in the Political Process 



Means (combined across clc>;;ses) 





No Game 


Game 


Combined 


No Role 


+ .17 


+ .13 


+ .15 


Role 


+ .17 


+ .18 


+ .18 


Combined 


+ .17 


+ .15 


+ .16 


Standard deviation. 


within cells (pooled) = 


.31 





Internal consistency (alpha) = .43 



Analysis of variance 



Source 


df 


Game 


1 


Role 


1. 


Classes 


3 


GxR 


1 


GxC 


3 


RxC 


3 


GxRxC 


3 


Within Cells 


125 


Total 


140 



SS 


MS 


F 


P 


0.01 


0.01 


0.14 


N.S. 


0.03 


0.03 


0.27 


N.S. 


0.35 


0.12 


1.20 


N.S. 


0.03 


0.03 


0.28 


N.S. 


0.18 


0.06 


0.64 


N.S. 


0.94 


0.31 


3.29 


.025 


0.60 


0.02 


2.09 


N.S. 


11.97 


0.096 






14.12 










.067 
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Table 5 



Knowledge of Names of Own Congressmen 





Means (combined across 


classes) 




No Game 


Game 


Comb ined 


No Role 


.76 


.60 


.68 


Role 


.62 


.51 


.57 


Combined 


.69 


.56 


.62 


Standard 


deviation, within cells (pooled) = 


.32 




Internal 


consistency (alpha) = .68 







Analysis of variance 



Source 


df 


SS 


MS 


F 


P 




Game 


1 


0.56 


0.56 


5.48 


.025 


.030 


Role 


1 


0.45 


0.45 


4.40 


.05 


.024 


Classes 


3 


1.03 


0.34 


3.36 


.025 


.055 


GxR 


1 


0.04 


0.04 


0.35 


N.S. 




GxC 


3 


0.43 


0.14 


1.41 


N.S. 




RxC 


3 


2.41 


0.80 


7.90 


.001 


.130 


GxRxC 


3 


0.88 


0.29 


3.20 


.05 


.04 7 


Within cells 


125 


12.71 


0.102 








Total 


140 


18.49 
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Note: Numbers below the main diagonal are pooled, within-cell correlations • 

Numbers on the main diagonal are internal-consistency estimates. No 
Internal -consistency estimate is possible for "Belief in prevalence 
of log-rolling," since it was measured by only a single questionnaire 
item. 



